We wish to learn a hiddend-dimensional valuew . Every round, we are presented with a subset Xt Rd of possible actions.

Wedesign a novel truncation operation to further reduce the computational complexity of theproposed algorithms.

Consider the following problem faced by a geographical query service (e.g.

Active learning methods aim to improve sample complexity in machine learning. In this work, we investigate an active learning scheme via a novel gradient-free cutting-plane training method for ReLU networks of arbitrary depth. We demonstrate, for the first time, that cutting-plane algorithms, traditionally used in linear models, can be extended to deep neural networks despite their nonconvexity and nonlinear decision boundaries. Our results demonstrate that these methods provide a promising alternative to the commonly employed gradient-based optimization techniques in large-scale neural networks. Moreover, this training method induces the first deep active learning scheme known to achieve convergence guarantees. We exemplify the effectiveness of our proposed active learning method against popular deep active learning baselines via both synthetic data experiments and sentimental classification task on real datasets.

However, it is notoriously di fficult to solve in practice, because IPMs memory requirements scale at a demanding rate. Indeed, state-of-the-art SDO solvers such as MOSEK cannot solve constrained instances of Problem (1) with n 250 variables on a standard laptop, and it is optimization folklore that there is a gap between SDOs theoretical and practical tractability. Motivated by the demanding memory requirements of IPMs, a stream of literature studies inexact methods for SDOs, which replace the semidefinite constraint with weaker yet less computationally demanding constraints. This approach was first investigated by Kim and Kojima [13], who observed that relaxing a positive semidefinite constraint to the weaker constraint that all 2 2 minors of a matrix are positive semidefinite yields a second order cone (SOC)-representable outer approximation of the positive semidefinite (PSD) cone. In a related line of work, Krishnan and Mitchell [15] propose applying Kelley [12]'s cutting plane method to generate

Optimization problems are everywhere in engineering: Balancing design tradeoffs is an optimization problem, as are scheduling and logistical planning. The theory -- and sometimes the implementation -- of control systems relies heavily on optimization, and so does machine learning, which has been the basis of most recent advances in artificial intelligence. This week, at the IEEE Symposium on Foundations of Computer Science, a trio of present and past MIT graduate students won a best-student-paper award for a new "cutting-plane" algorithm, a general-purpose algorithm for solving optimization problems. The algorithm improves on the running time of its most efficient predecessor, and the researchers offer some reason to think that they may have reached the theoretical limit. But they also present a new method for applying their general algorithm to specific problems, which yields huge efficiency gains -- several orders of magnitude.